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LONG-TERM  GOALS 

A  significant  source  of  uncertainty  in  the  prediction  of  the  apparent  optical  properties  of  the  ocean  is 
the  geometrical  distribution  of  the  radiance  field  and  its  variation  with  respect  to  time  and  space;  this 
uncertainty  directly  affects  attempts  to  use  measurements  of  reflectance  and  attenuation  for  the 
diagnosis  of  ocean  constituents.  Uncertainties  in  the  time  and  depth  dependent  variations  in  the 
radiance  distribution,  and  their  sources  of  variation,  propagate  as  well  to  the  prediction  of  the 
performance  of  new  imaging  systems  such  as  the  “virtual  periscope”.  The  problem  starts  at  the  sea 
surface,  where  the  generally  unknown  sky  radiance  distribution,  coupled  with  a  roughened  air-sea 
interface,  plays  a  major  role  in  the  transmission  of  sun  and  sky  radiance  to  below  the  surface.  In  the 
ocean  interior,  the  volume  scattering  function,  and  the  absorption  coefficient  alter  the  radiance 
distribution  in  both  the  forward  and  backward  direction;  in  the  perhaps  usual  situation  of  multiple 
scattering,  the  uncertainty  in  the  radiance  distribution  becomes  large.  In  optically  shallow  areas,  non- 
Lambertian  bottom  reflectances  add  to  the  uncertainty. 

Our  long-tenn  goal  is  to  develop  and  deploy  a  relatively  simple  means  for  the  measurement  of  the  full 
radiance  distribution,  which  could  be  routinely  deployed  by  the  optical  oceanographic  community.  A 
further  side  benefit  would  be  that  many  of  the  measurements  currently  made,  such  as  planar  and  scalar 
irradiance,  angle-dependent  Q  factor  etc.,  could  be  made  by  various  integration  operations  on  the 
measured  radiance  field  rather  than  with  mechanical  diffusers.  The  potential  interferences  of  various 
deployment  platforms  (e.g.  shading,  reflectances  by  ships,  buoys  and  towers)  could  be  measured 
directly  rather  than  inferred  based  on  inaccurate  assumptions  about  the  underwater  radiance 
distribution.  A  direct  confirmation  of  the  asymptotic  radiance  distribution  can  be  made.  Finally,  high 
quality  quantitative  (and  radiometrically  calibrated)  measurements  of  the  radiance  distribution,  and 
their  time  and  depth  derivatives,  can  in  principle  (but  not  yet  in  practice)  be  used  to  estimate  all  the 
inherent  optical  properties  (both  absorption  and  volume  scattering  coefficient)  and  as  well  the  nature  of 
the  air-sea  interface. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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OBJECTIVES 


The  Radiance  Camera  or  RadCam  project  is  part  of  the  Radiance  in  a  Dynamic  Ocean  (RaDyO) 
program.  The  primary  objective  is  to  create  a  camera  that  can  record  the  spatial  radiance  distribution 
at  the  ocean  surface  and  at  depth.  The  proposed  instrument  will  be  uniquely  capable  of  resolving  both 
the  downwelling  and  upwelling  radiance  distribution  and  its  variation  with  depth,  time  and  wavelength 
(  L(z,  t,  6,  (/),a)  );  from  these  measurements,  the  apparent  optical  properties  ED,  Eu,  Ea,  Eou  and  E0d  are 
computed  by  integration.  The  distribution  functions  (e.g.  the  average  cosines)  are  computed  directly, 
as  are  the  various  diffuse  attenuation  coefficients  and  reflectances.  The  fully-specified  radiance  field 
therefore  provides  all  the  pertinent  information  to  derive  not  only  the  apparent  optical  properties,  but 
the  inherent  optical  properties:  the  absorption  coefficient  and,  in  principle  by  inversion,  the  volume 
scattering  function.  An  instrument  capable  of  this  measurement  to  the  necessary  accuracy,  resolution, 
and  noise  characteristics  could,  again  in  principle,  replace  all  or  most  of  the  optical  instruments 
currently  deployed  today.. 

APPROACH 

While  radiance  cameras  have  been  built  before,  they  have  not  been  able  to  image  the  sun  at  the  surface 
due  to  the  very  high  scene  dynamic  range.  RadCam  will  take  advantage  of  recent  developments  in 
high-dynamic  range  (HDR)  CMOS  imaging  arrays.  These  arrays  were  developed  for  science, 
surveillance,  and  automotive  applications.  Traditional  CCD  arrays  are  linear,  limiting  the  dynamic 
range  that  can  be  achieved.  These  HDR  CMOS  arrays  use  a  number  of  different  methods  to  produce  a 
nonlinear  response  function,  giving  scene  dynamic  ranges  of  up  to  120  dB  or  6  decades. 

WORK  COMPLETED 

In  the  first  year  of  this  project  we  considered  several  possible  cameras  and  imaging  arrays.  We  tested 
two  candidate  cameras/arrays  and  selected  one  for  RadCam.  Measurements  have  shown  that  it  can 
achieve  a  scene  dynamic  range  of  6  decades,  and  an  impressive  system  dynamic  range  of  nearly  10 
decades. 

Three  instruments  are  being  designed  as  part  of  this  project.  The  first  is  a  reference  camera  that  will  be 
mounted  on  deck.  The  second  is  a  logging-type  instrument  that  can  be  mounted  on  a  Bluefin  AUV  or 
an  ROV.  The  third  is  a  profiler  that  sends  data  to  the  surface  for  real-time  processing.  The  first  two 
cameras  are  upward  looking  only  (i.e.  they  record  downwelling  radiance)  while  the  profiler  has  both  an 
upwelling  and  downwelling  camera.  This  allows  it  to  measure  radiance  over  the  entire  sphere  around 
the  instrument. 

At  this  time,  only  the  first  two  cameras  are  being  built,  though  the  design  considers  the  profiler. 

RESULTS 

Optics 

A  fisheye  lens  with  a  specified  185  degree  field  of  view  was  selected  for  use  in  the  camera.  A  custom 
optical  system,  shown  in  Figure  1,  was  designed  to  reduce  the  image  size  produced  by  the  fisheye  to  fit 


2 


totally  within  the  CMOS  array.  The  system  also  contains  a  bandpass  filter  centered  at  555  nm  with  a 
bandwidth  of  20  nm. 


Fisheye  Triplet  Intermediate  Bandpass  Achromat  CMOS  Array 

Field  Lens  Image  Filter  Doublets 


Figure  1:  Custom  optical  system  used  to  reduce  image  size  and  spectrally  filter  the  image. 
It  includes  a  fisheye  lens,  field  lens,  filter,  a  pair  of  achromat  doublets  and  the  CMOS  array. 


The  geometric  mapping  of  the  optical  system  is  shown  in  Figure  2.  Most  fisheye  lenses  use  an 
equidistant  projection,  where  the  field  angle  maps  linearly  to  a  radial  position  in  the  image.  The 
measurements  show  a  field  angle  of  190  degrees  mapped  to  an  image  466  pixels  in  diameter.  This 
corresponds  to  an  angular  resolution  of  about  0.4  degrees/pixel,  better  than  the  1  degree/pixel  specified 
at  the  outset  of  the  project. 

The  rolloff  of  the  optical  system  is  shown  in  Figure  3.  The  measurements  show  a  drop  in  throughput 
to  92%  at  a  field  angle  of  95  degrees.  This  drop  is  almost  entirely  due  to  the  fisheye  lens.  Thus  the 
custom  optical  system  exhibits  excellent  performance. 


Field  Angle  [degrees] 

Figure  2:  Graph  of  the  geometric  projection  of  the  optical  system  (image  radial 
position  versus  field  angle)  showing  that  the  relationship  is  very  linear. 


3 


Figure  3:  Graph  showing  the  rolloff  of  the  optical  system.  The  rolloff  is 
negligible  out  to  about  60  degrees  field  angle  and  drops  to  about  92%  at  95  degrees. 


Figure  4  and  Figure  5  show  the  assembled  optical  system,  including  the  camera  electronics. 


Figure  4:  Side  view  of  the  camera  assembly  showing  the  fisheye  lens, 
custom  optical  system,  a  translator  for  accurately  positioning  the  image 
on  the  array,  and  camera  electronics.  The  whole  assembly  is  16  cm  long. 
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Figure  5:  Picture  of  the  backside  view  of  the  custom  camera 
circuit  board.  The  metallic  box  is  the  fiber  transceiver. 


Electronics 

Custom  electronics  were  required  to  serialize  the  video  stream  from  the  camera  (Profiler  and 
ROV/AUV)  and  transmit  it  over  a  fiber  optic  cable  to  a  computer  on-deck  for  real  time  processing. 

The  serialized  data  rate  is  approximately  500  Mb/s.  At  the  surface  the  data  is  deserialized  and  sent  to  a 
framegrabber.  The  framegrabber  assembles  the  images  and  transfers  them  to  memory  for  processing. 
The  boards  have  been  designed,  built,  and  tested,  with  the  exception  of  the  connection  to  the 
framegrabber.  The  custom  camera  board  was  designed  such  that  it  can  be  used  in  any  of  the 
instruments. 

The  instruments  were  designed  to  handle  data  from  ancillary  sensors  including  tilt/heading  and 
pressure.  The  profiler  will  include  a  full  suite  of  instruments  including  integrating  radiometers, 
attenuation  and  backscatter  meters  and  others.  The  data  from  these  instruments  is  interleaved  with  the 
video  data  stream. 

The  ROV/AUV  camera  has  no  connection  to  a  deck  unit  and  logs  data  onboard.  A  PC  104-style 
computer  captures  and  stores  the  video  and  ancillary  sensor  data.  On  retrieval  of  the  instrument,  the 
data  is  offloaded  using  a  high-speed  ethernet  connection.  The  only  real-time  processing  perfonned  on¬ 
board  is  determining  which  exposure  to  use. 

Software 

Much  of  the  instrument  control  software  has  already  been  written  for  the  cameras.  It  is  written  in  Java 
with  interfaces  to  native  C  code  for  specific  third-party  hardware.  Java  simplifies  the  development  of  a 
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user  interface  including  displaying  live  video  and  camera  control.  It  will  interface  with  existing 
Satlantic  code  to  display  and  log  ancillary  sensor  data. 


Images  will  be  processed  in  real  time,  though  the  maximum  frame  rate  will  depend  on  the  processing 
power  of  the  computer  and  still  needs  to  be  determined.  Processing  includes  applying  calibration 
coefficients  to  each  pixel,  compensating  for  tilt/heading,  and  calculating  apparent  and  inherent  optical 
properties  (see  Figure  6).  The  software  must  also  determine  when  a  change  in  exposure  is  required. 
This  last  task  is  not  trivial  and  has  not  yet  been  implemented. 


Figure  6:  Test  image  (left)  and  output  (right)  of  the  tilt/heading  compensation 
algorithm.  User  supplied  tilt  and  heading  information  were  used  instead  of  real 
sensor  data,  for  testing  purposes.  The  compensated  image  shows  a  crescent  shaped 
area  that  corresponds  to  a  region  outside  the  fisheye  field  of  view. 


Calibration 

The  calibration  of  the  CMOS  array  is  one  of  the  most  challenging  parts  of  the  RadCam  project.  Unlike 
CCDs  which  have  a  quite  uniform  response  over  the  array,  every  pixel  in  a  CMOS  device  has  a 
slightly  different  response  (grayscale  versus  radiance).  This  is  because  each  pixel  in  the  CMOS  array 
contains  its  own  electronics.  Thus  every  pixel  in  the  array  must  be  individually  calibrated. 

Particular  challenges  for  calibrating  RadCam  are  the  very  large  dynamic  range  and  the  very  high  peak 
radiance  (7*1 06  uW/cm2/nm/sr).  Two  methods  of  measuring  the  response  curves  were  used.  The  first 
is  a  standard  method  that  uses  a  NIST  traceable  FEL  lamp,  a  rail,  and  neutral  density  filters.  Images 
are  captured  from  the  array  at  several  distances  from  the  lamp.  The  limited  power  and  point  source 
nature  of  the  FEL  lamp  do  not  allow  it  to  be  used  for  the  highest  end  of  the  response  curve.  An  arc 
lamp  was  tested  but  a  beam  of  sufficient  spatial  uniformity  could  not  be  produced.  We  now  use  a 
HeNe  laser  as  the  source  and  scan  the  array  behind  it.  While  the  beam  from  a  HeNe  is  Gaussian,  it 
was  found  to  be  sufficiently  unifonn  over  a  small  spot  (few  tens  of  microns  in  diameter)  that  it  could 
be  used  for  calibration.  During  development  of  the  calibration  procedure  a  nonlinearity  in  the  array 


6 


was  observed.  At  high  radiances  the  grayscale  response  of  the  array  can  saturate  and  then  decrease  - 
an  effect  the  manufacturer  describes  as  eclipsing.  On-chip  circuitry  is  used  to  combat  the  problem  and 
works  well  when  small  numbers  of  pixels  are  brightly  illuminated.  When  large  numbers  of  pixels  are 
brightly  illuminated  the  problem  is  still  observed.  Only  solar  and  nearby  pixels  should  reach  these 
radiance  levels,  so  the  final  images  should  be  accurate,  though  some  verification  will  be  perfonned. 

Software  was  written  to  semi-automate  data  collection  during  calibration.  It  takes  care  of  scanning  the 
array  behind  the  HeNe,  isolating  relevant  data,  and  averaging  frames  together  to  reduce  temporal  noise. 
Software  was  also  written  to  process  the  large  calibration  data  sets.  The  program  takes  care  of  fitting 
together  data  runs,  sectioning  the  nonlinear  response  curves,  and  fitting  polynomials  and  hyperbolics  to 
the  sections.  A  final  calibration  coefficient  file  is  about  30  MB  per  exposure  setting. 
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Figure  7:  Calibration  data  showing  runs,  fitted  together  to  produce  an  overall  response  curve. 
At  the  left  end,  there  are  some  fitting  problems  believed  to  be  due  to  stray  light  during 
data  capture.  Data  for  the  high-radiance  end  of  this  exposure  is  not  shown. 


While  the  system  will  operate  over  nearly  10  decades  of  dynamic  range,  only  about  6  decades  can  be 
captured  using  any  particular  exposure.  Therefore,  a  search  was  performed  to  determine  which 
exposure  settings  would  be  suitable.  We  have  decided  on  three  exposures  that  cover  a  dynamic  range 
of  more  than  9  decades.  The  roughly  measured  curves  are  shown  in  Figure  6. 
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Figure  8:  Graph  showing  roughly-measured  response  curves  (grayscale  versus  relative  radiance) 
for  three  different  exposures  that  will  be  used  in  RadCam.  The  first  exposure,  suitable  for  use  at  the 
surface,  has  about  6  decades  dynamic  range.  The  second  exposure  extends  from  about  -3  decades 
to  nearly  -9  decades.  The  third  exposure  extends  from  about  -6.5  decades  to  about  -9.5  decades. 


Mechanical 

Figure  9  shows  the  basic  layout  of  two  of  the  RadCam  instruments;  the  ROV/AUV  camera  and  the 
Reference  camera.  The  ROV/AUV  camera  is  wider  to  accommodate  a  PC  1 04  computer,  while 
conforming  to  the  space  limitations  of  a  Blucfin  AUV  instrument  bay.  There  are  some  additional 
components  that  will  be  in  the  instrument,  including  a  pressure  sensor,  that  are  not  shown.  The 
Reference  camera  is  shown  with  a  dome  but  will  be  operated  without  the  dome  eliminate  reflections 
and  glare. 


Figure  9:  Mechanical  layout  of  the  ROV/AUV 
camera  (left)  and  the  Reference  camera  (right). 
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IMPACT/APPLICATIONS 


The  camera  may  have  applications  for  various  sorts  of  surveillance.  The  derivation  of  optical 
properties  from  measurement  of  the  full  radiance  distribution  may  have  practical  applications. 

RELATED  PROJECTS 

This  project  is  embedded  within  the  Radiance  in  a  Dynamic  Ocean  (RaDyO)  program,  and  hence  is 
related  to  all  projects  contained  therein. 
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